{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 项目简介\n",
    "该项目使用PaddleX快速训练垃圾分类模型，然后通过PaddleLite部署到华为Mate20 Pro手机上，实现飞桨框架深度学习模型的落地。\n",
    "\n",
    "- 模型训练：PaddleX，`YoloV3`的`backbone`使用`MobileNetV3_large`和`DarkNet53`\n",
    "- 模型转换：Paddle-Lite\n",
    "- Android开发环境：Android Studio on Ubuntu 18.04 64-bit\n",
    "- 移动端设备：华为Mate20 Pro\n",
    "\n",
    "## 关于本项目\n",
    "> 针对项目还存在的改进空间，以及其它模型在不同移动设备的部署，希望大家多交流观点、介绍经验，共同学习进步。[个人主页](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/90149)\n",
    "# PaddleX快速训练迁移学习模型\n",
    "## PaddleX简介\n",
    "> [PaddleX](https://gitee.com/paddlepaddle/PaddleX)是基于飞桨核心框架、开发套件和工具组件的深度学习全流程开发工具。具备全流程打通、融合产业实践、易用易集成三大特点。\n",
    "> \n",
    "> [PaddleX文档](https://paddlex.readthedocs.io/)\n",
    "## 安装工具库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!pip install paddlelite\n",
    "!pip install paddlex\n",
    "!pip install shapely"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 准备数据集\n",
    "本项目为目标检测场景，使用的垃圾分类数据集标注为COCO格式，包含图像文件夹及图像标注信息文件。\n",
    "参考数据文件结构如下：\n",
    "```\n",
    "./dataset/  # 数据集根目录\n",
    "|--JPEGImages  # 图像目录\n",
    "|  |--xxx1.jpg\n",
    "|  |--...\n",
    "|  └--...\n",
    "|\n",
    "|--train.json  # 训练相关信息文件\n",
    "|\n",
    "└--val.json  # 验证相关信息文件\n",
    "\n",
    "```\n",
    "其中，相应的文件名可根据需要自行定义。\n",
    "\n",
    "`train.json`和`val.json`存储与标注信息、图像文件相关的信息。如下所示：\n",
    "\n",
    "```\n",
    "{\n",
    "  \"annotations\": [\n",
    "    {\n",
    "      \"iscrowd\": 0,\n",
    "      \"category_id\": 1,\n",
    "      \"id\": 1,\n",
    "      \"area\": 33672.0,\n",
    "      \"image_id\": 1,\n",
    "      \"bbox\": [232, 32, 138, 244],\n",
    "      \"segmentation\": [[32, 168, 365, 117, ...]]\n",
    "    },\n",
    "    ...\n",
    "  ],\n",
    "  \"images\": [\n",
    "    {\n",
    "      \"file_name\": \"xxx1.jpg\",\n",
    "      \"height\": 512,\n",
    "      \"id\": 267,\n",
    "      \"width\": 612\n",
    "    },\n",
    "    ...\n",
    "  ]\n",
    "  \"categories\": [\n",
    "    {\n",
    "      \"name\": \"labelA\",\n",
    "      \"id\": 1,\n",
    "      \"supercategory\": \"component\"\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "```\n",
    "其中，每个字段的含义如下所示：\n",
    "\n",
    "| 域名 | 字段名 | 含义 | 数据类型 | 备注 |\n",
    "|:-----|:--------|:------------|------|:-----|\n",
    "| annotations | id | 标注信息id | int | 从1开始 |\n",
    "| annotations | iscrowd      | 标注框是否为一组对象 | int | 只有0、1两种取值 |\n",
    "| annotations | category_id  | 标注框类别id | int |  |\n",
    "| annotations | area         | 标注框的面积 | float |  |\n",
    "| annotations | image_id     | 当前标注信息所在图像的id | int |  |\n",
    "| annotations | bbox         | 标注框坐标 | list | 长度为4，分别代表x,y,w,h |\n",
    "| annotations | segmentation | 标注区域坐标 | list | list中有至少1个list，每个list由每个小区域坐标点的横纵坐标(x,y)组成 |\n",
    "| images          | id                | 图像id | int | 从1开始 |\n",
    "| images   | file_name         | 图像文件名 | str |  |\n",
    "| images      | height            | 图像高度 | int |  |\n",
    "| images       | width             | 图像宽度 | int |  |\n",
    "| categories  | id            | 类别id | int | 从1开始 |\n",
    "| categories | name          | 类别标签名 | str |  |\n",
    "| categories | supercategory | 类别父类的标签名 | str |  |\n",
    "\n",
    "其他格式数据准备方式请参考官方文档：\n",
    "[数据集格式说明](https://gitee.com/paddlepaddle/PaddleX/blob/develop/docs/appendix/datasets.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-07-05 11:12:45--  https://bj.bcebos.com/paddlex/datasets/garbage_ins_det.tar.gz\n",
      "Resolving bj.bcebos.com (bj.bcebos.com)... 182.61.200.229, 182.61.200.195\n",
      "Connecting to bj.bcebos.com (bj.bcebos.com)|182.61.200.229|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 196078673 (187M) [application/octet-stream]\n",
      "Saving to: ‘garbage_ins_det.tar.gz’\n",
      "\n",
      "garbage_ins_det.tar 100%[===================>] 187.00M  30.0MB/s    in 11s     \n",
      "\n",
      "2020-07-05 11:12:56 (17.3 MB/s) - ‘garbage_ins_det.tar.gz’ saved [196078673/196078673]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "! wget https://bj.bcebos.com/paddlex/datasets/garbage_ins_det.tar.gz\n",
    "! tar xzf garbage_ins_det.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 模型训练\n",
    "### 配置GPU环境"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['CUDA_VISIBLE_DEVICES'] = '0'\n",
    "import paddlex as pdx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 数据处理与数据增强\n",
    "更多详细的数据增强组合请参考[官方文档](https://gitee.com/paddlepaddle/PaddleX/blob/develop/docs/apis/transforms/det_transforms.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from paddlex.det import transforms\n",
    "train_transforms = transforms.Compose([\n",
    "    transforms.RandomHorizontalFlip(),\n",
    "    transforms.Normalize(),\n",
    "    transforms.ResizeByShort(short_size=512, max_size=512),\n",
    "    transforms.Padding(coarsest_stride=32)\n",
    "])\n",
    "\n",
    "eval_transforms = transforms.Compose([\n",
    "    transforms.Normalize(),\n",
    "    transforms.ResizeByShort(short_size=512, max_size=512),\n",
    "    transforms.Padding(coarsest_stride=32),\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 创建数据读取器\n",
    "关于其它格式目标检测数据集的读取方式请参考[检测和实例分割数据集](https://gitee.com/paddlepaddle/PaddleX/blob/develop/docs/apis/datasets/detection.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loading annotations into memory...\n",
      "Done (t=0.01s)\n",
      "creating index...\n",
      "index created!\n",
      "2020-07-06 20:44:57 [INFO]\tStarting to read file list from dataset...\n",
      "2020-07-06 20:44:57 [INFO]\t221 samples in file garbage_ins_det/train.json\n",
      "loading annotations into memory...\n",
      "Done (t=0.00s)\n",
      "creating index...\n",
      "index created!\n",
      "2020-07-06 20:44:57 [INFO]\tStarting to read file list from dataset...\n",
      "2020-07-06 20:44:57 [INFO]\t62 samples in file garbage_ins_det/val.json\n"
     ]
    }
   ],
   "source": [
    "train_dataset = pdx.datasets.CocoDetection(\n",
    "    data_dir='garbage_ins_det/JPEGImages',\n",
    "    ann_file='garbage_ins_det/train.json',\n",
    "    transforms=train_transforms,\n",
    "    shuffle=True)\n",
    "eval_dataset = pdx.datasets.CocoDetection(\n",
    "    data_dir='garbage_ins_det/JPEGImages',\n",
    "    ann_file='garbage_ins_det/val.json',\n",
    "    transforms=eval_transforms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 使用`MobileNetV3_large`做`backbone`训练YoloV3迁移学习模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "num_classes = len(train_dataset.labels)\r\n",
    "model = pdx.det.YOLOv3(num_classes=num_classes, backbone='MobileNetV3_large')\r\n",
    "model.train(\r\n",
    "    num_epochs=270,\r\n",
    "    train_dataset=train_dataset,\r\n",
    "    train_batch_size=8,\r\n",
    "    eval_dataset=eval_dataset,\r\n",
    "    learning_rate=0.000125,\r\n",
    "    lr_decay_epochs=[210, 240],\r\n",
    "    save_dir='output/yolov3_mobilevetv3',\r\n",
    "    use_vdl=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 模型效果评估\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/911351e30dd44ca187a2edf60259f3ac341eb627dfcb4e979fc48f8c197ee59a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "eval_details_file = 'output/yolov3_mobilevetv3/epoch_270/eval_details.json'\n",
    "pdx.det.draw_pr_curve(eval_details_file, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-07-06 20:45:52 [INFO]\tModel[YOLOv3] loaded.\n",
      "loading annotations into memory...\n",
      "Done (t=0.00s)\n",
      "creating index...\n",
      "index created!\n",
      "2020-07-06 20:45:52 [INFO]\tStarting to read file list from dataset...\n",
      "2020-07-06 20:45:52 [INFO]\t31 samples in file garbage_ins_det/test.json\n",
      "2020-07-06 20:45:52 [INFO]\tStart to evaluating(total_samples=31, total_steps=4)...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4/4 [00:10<00:00,  2.65s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.06s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.03s).\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286\n",
      " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.741\n",
      " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.192\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.288\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.243\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.390\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.390\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.394\n",
      "creating index...\n",
      "index created!\n",
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.04s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.01s).\n"
     ]
    }
   ],
   "source": [
    "model = pdx.load_model('output/yolov3_mobilevetv3/epoch_270')\n",
    "eval_dataset = pdx.datasets.CocoDetection(\n",
    "    data_dir='garbage_ins_det/JPEGImages',\n",
    "    ann_file='garbage_ins_det/test.json',\n",
    "    transforms=eval_transforms)\n",
    "metrics, evaluate_details = model.evaluate(eval_dataset, batch_size=8, return_details=True)\n",
    "gt = evaluate_details['gt']\n",
    "bbox = evaluate_details['bbox']\n",
    "pdx.det.draw_pr_curve(gt=gt, pred_bbox=bbox, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%mkdir log\n",
    "%cp output/yolov3_mobilevetv3/vdl_log/*.log log/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "查看VisualDL训练过程，需要执行如下步骤：\n",
    "1. 将`output/yolov3_mobilevetv3/vdl_log`目录下的`.log`文件移动到`/home/aistudio/log`目录下\n",
    "2. 打开终端执行命令`visualdl --logdir ./log --port 8001`\n",
    "3. 复制本项目的网址并将`notebooks`之后内容全部替换为`visualdl`，如打开网页`https://aistudio.baidu.com/bdvgpu32g/user/90149/613622/visualdl`\n",
    "\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/1901479b637a4df9a7528a10a0f70e333d5daa081f134c0ba8acbe0a4842e6fd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 使用`DarkNet53`做`backbone`训练YoloV3迁移学习模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "num_classes = len(train_dataset.labels)\n",
    "model = pdx.det.YOLOv3(num_classes=num_classes, backbone='DarkNet53')\n",
    "model.train(\n",
    "    num_epochs=270,\n",
    "    train_dataset=train_dataset,\n",
    "    train_batch_size=8,\n",
    "    eval_dataset=eval_dataset,\n",
    "    learning_rate=0.000125,\n",
    "    lr_decay_epochs=[210, 240],\n",
    "    save_dir='output/yolov3_darknet53',\n",
    "    use_vdl=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 模型效果评估\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/484a0e622e1b4699b535f85e7b4a908b18cb457477bd4184afc05447572e5f67)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "eval_details_file = 'output/yolov3_darknet53/epoch_270/eval_details.json'\n",
    "pdx.det.draw_pr_curve(eval_details_file, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-07-06 20:47:48 [INFO]\tModel[YOLOv3] loaded.\n",
      "loading annotations into memory...\n",
      "Done (t=0.00s)\n",
      "creating index...\n",
      "index created!\n",
      "2020-07-06 20:47:48 [INFO]\tStarting to read file list from dataset...\n",
      "2020-07-06 20:47:48 [INFO]\t31 samples in file garbage_ins_det/test.json\n",
      "2020-07-06 20:47:48 [INFO]\tStart to evaluating(total_samples=31, total_steps=4)...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4/4 [00:03<00:00,  1.32it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.07s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.02s).\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.310\n",
      " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.773\n",
      " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.211\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.312\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.245\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.403\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.406\n",
      "creating index...\n",
      "index created!\n",
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.05s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.01s).\n"
     ]
    }
   ],
   "source": [
    "model = pdx.load_model('output/yolov3_darknet53/epoch_270')\n",
    "eval_dataset = pdx.datasets.CocoDetection(\n",
    "    data_dir='garbage_ins_det/JPEGImages',\n",
    "    ann_file='garbage_ins_det/test.json',\n",
    "    transforms=eval_transforms)\n",
    "metrics, evaluate_details = model.evaluate(eval_dataset, batch_size=8, return_details=True)\n",
    "gt = evaluate_details['gt']\n",
    "bbox = evaluate_details['bbox']\n",
    "pdx.det.draw_pr_curve(gt=gt, pred_bbox=bbox, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 训练FasterRCNN模型（转换Paddle-Lite模型不成功，没有实际部署）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "num_classes = len(train_dataset.labels) + 1\n",
    "model = pdx.det.FasterRCNN(num_classes=num_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model.train(\n",
    "    num_epochs=12,\n",
    "    train_dataset=train_dataset,\n",
    "    train_batch_size=2,\n",
    "    eval_dataset=eval_dataset,\n",
    "    learning_rate=0.0025,\n",
    "    lr_decay_epochs=[8, 11],\n",
    "    save_dir='output/faster_rcnn_r50_fpn',\n",
    "    use_vdl=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 模型效果评估\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/9e138e484eae4d9d9c155018f706c5d5d7f66e5e59b546c0a546776fb6ac0cc7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "eval_details_file = 'output/faster_rcnn_r50_fpn/epoch_12/eval_details.json'\n",
    "pdx.det.draw_pr_curve(eval_details_file, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-07-06 20:49:07 [INFO]\tModel[FasterRCNN] loaded.\n",
      "loading annotations into memory...\n",
      "Done (t=0.00s)\n",
      "creating index...\n",
      "index created!\n",
      "2020-07-06 20:49:07 [INFO]\tStarting to read file list from dataset...\n",
      "2020-07-06 20:49:07 [INFO]\t31 samples in file garbage_ins_det/test.json\n",
      "2020-07-06 20:49:07 [WARNING]\tFaster RCNN supports batch_size=1 only during evaluating, so batch_size is forced to be set to 1.\n",
      "2020-07-06 20:49:07 [INFO]\tStart to evaluating(total_samples=31, total_steps=31)...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 31/31 [00:04<00:00,  6.82it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.06s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.03s).\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.867\n",
      " Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 1.000\n",
      " Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 1.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.667\n",
      " Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.869\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.509\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.885\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.885\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.700\n",
      " Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.888\n",
      "creating index...\n",
      "index created!\n",
      "creating index...\n",
      "index created!\n",
      "Running per image evaluation...\n",
      "Evaluate annotation type *bbox*\n",
      "DONE (t=0.03s).\n",
      "Accumulating evaluation results...\n",
      "DONE (t=0.01s).\n"
     ]
    }
   ],
   "source": [
    "model = pdx.load_model('output/faster_rcnn_r50_fpn/epoch_12')\n",
    "eval_dataset = pdx.datasets.CocoDetection(\n",
    "    data_dir='garbage_ins_det/JPEGImages',\n",
    "    ann_file='garbage_ins_det/test.json',\n",
    "    transforms=eval_transforms)\n",
    "metrics, evaluate_details = model.evaluate(eval_dataset, batch_size=8, return_details=True)\n",
    "gt = evaluate_details['gt']\n",
    "bbox = evaluate_details['bbox']\n",
    "pdx.det.draw_pr_curve(gt=gt, pred_bbox=bbox, save_dir='./garbage_ins_det')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 模型部署准备\n",
    "## Paddle-Lite简介\n",
    "> [Paddle Lite](https://gitee.com/paddlepaddle/paddle-lite)为Paddle-Mobile的升级版，定位支持包括手机移动端在内更多场景的轻量化高效预测，支持更广泛的硬件和平台，是一个高性能、轻量级的深度学习预测引擎。在保持和PaddlePaddle无缝对接外，也兼容支持其他训练框架产出的模型。\n",
    "> \n",
    "> 完整使用文档位于 [PaddleLite 文档](https://paddle-lite.readthedocs.io/zh/latest/) 。\n",
    "## 导出inference模型\n",
    "\n",
    "参考PaddleX文档：在服务端部署的模型需要首先将模型导出为inference格式模型，导出的模型将包括`__model__`、`__params__`和`model.yml`三个文名，分别为模型的网络结构，模型权重和模型的配置文件（包括数据预处理参数等等）。在安装完PaddleX后，在命令行终端使用如下命令导出模型到当前目录`inferece_model`下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W0706 20:52:39.260469  1399 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.0\n",
      "W0706 20:52:39.264473  1399 device_context.cc:260] device: 0, cuDNN Version: 7.3.\n",
      "2020-07-06 20:52:42 [INFO]\tModel[YOLOv3] loaded.\n",
      "2020-07-06 20:52:43 [INFO]\tModel for inference deploy saved in ./inference_model/yolov3_mobilevetv3.\n"
     ]
    }
   ],
   "source": [
    "!paddlex --export_inference --model_dir=output/yolov3_mobilevetv3/epoch_270 --save_dir=./inference_model/yolov3_mobilevetv3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W0705 13:38:49.321007  1546 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 9.2, Runtime API Version: 9.0\n",
      "W0705 13:38:49.325225  1546 device_context.cc:260] device: 0, cuDNN Version: 7.3.\n",
      "2020-07-05 13:38:53 [INFO]\tModel[YOLOv3] loaded.\n",
      "2020-07-05 13:38:55 [INFO]\tModel for inference deploy saved in ./inference_model/yolov3_darknet53.\n"
     ]
    }
   ],
   "source": [
    "!paddlex --export_inference --model_dir=output/yolov3_darknet53/epoch_270 --save_dir=./inference_model/yolov3_darknet53"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W0706 21:16:32.794881  1451 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.0\n",
      "W0706 21:16:32.798995  1451 device_context.cc:260] device: 0, cuDNN Version: 7.3.\n",
      "2020-07-06 21:16:36 [INFO]\tModel[FasterRCNN] loaded.\n",
      "2020-07-06 21:16:37 [INFO]\tModel for inference deploy saved in ./inference_model/faster_rcnn_r50_fpn.\n"
     ]
    }
   ],
   "source": [
    "!paddlex --export_inference --model_dir=output/faster_rcnn_r50_fpn/epoch_12 --save_dir=./inference_model/faster_rcnn_r50_fpn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 转换PaddleLite模型\n",
    "这里没有使用PaddleX的deploy模块，会出现报错，具体原因待查。由于前面导出的模型是combined形式：模型文件夹model_dir下只有一个模型文件__model__和一个参数文件__params__；参考PaddleLite的文档，执行opt转换时只需要传入模型文件和参数文件路径。\n",
    "\n",
    "官方文档给出的***详尽的转化命令***总结\n",
    "```shell\n",
    "paddle_lite_opt \\\n",
    "    --model_dir=<model_param_dir> \\\n",
    "    --model_file=<model_path> \\\n",
    "    --param_file=<param_path> \\\n",
    "    --optimize_out_type=(protobuf|naive_buffer) \\\n",
    "    --optimize_out=<output_optimize_model_dir> \\\n",
    "    --valid_targets=(arm|opencl|x86|npu|xpu) \\\n",
    "    --record_tailoring_info =(true|false)\n",
    "```\n",
    "\n",
    "| 选项         | 说明 |\n",
    "| ------------------- | ------------------------------------------------------------ |\n",
    "| --model_dir         | 待优化的PaddlePaddle模型（非combined形式）的路径 |\n",
    "| --model_file        | 待优化的PaddlePaddle模型（combined形式）的网络结构文件路径。 |\n",
    "| --param_file        | 待优化的PaddlePaddle模型（combined形式）的权重文件路径。 |\n",
    "| --optimize_out_type | 输出模型类型，目前支持两种类型：protobuf和naive_buffer，其中naive_buffer是一种更轻量级的序列化/反序列化实现。若您需要在mobile端执行模型预测，请将此选项设置为naive_buffer。默认为protobuf。 |\n",
    "| --optimize_out      | 优化模型的输出路径。                                         |\n",
    "| --valid_targets     | 指定模型可执行的backend，默认为arm。目前可支持x86、arm、opencl、npu、xpu，可以同时指定多个backend(以空格分隔)，Model Optimize Tool将会自动选择最佳方式。如果需要支持华为NPU（Kirin 810/990 Soc搭载的达芬奇架构NPU），应当设置为npu, arm。 |\n",
    "| --record_tailoring_info | 当使用 [根据模型裁剪库文件](./library_tailoring.html) 功能时，则设置该选项为true，以记录优化后模型含有的kernel和OP信息，默认为false。 |\n",
    "\n",
    "* 如果待优化的fluid模型是非combined形式，请设置`--model_dir`，忽略`--model_file`和`--param_file`。\n",
    "* 如果待优化的fluid模型是combined形式，请设置`--model_file`和`--param_file`，忽略`--model_dir`。\n",
    "* 优化后的模型为以`.nb`名称结尾的单个文件。\n",
    "* 删除`prefer_int8_kernel`的输入参数，`opt`自动判别是否是量化模型，进行相应的优化操作。\n",
    "\n",
    "关于[python调用opt转化模型](https://gitee.com/paddlepaddle/paddle-lite/blob/develop/docs/user_guides/opt/opt_python.md)的详细参数设置介绍，可以参考官方文档。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 统计模型算子信息、判断是否支持\n",
    "> 注意：即使显示支持模型转换，实际也不一定能转换成功，比如该项目中的`faster_rcnn_r50_fpn`模型，具体原因待查"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!paddle_lite_opt --print_model_ops=true  --model_dir=inference_model/yolov3_mobilevetv3 --valid_targets=arm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!paddle_lite_opt --print_model_ops=true  --model_dir=inference_model/yolov3_darknet53 --valid_targets=npu,arm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OPs in the input model include:\r\n",
      "                                        OP_name      Host       X86      CUDA       ARM    OpenCL      FPGA       NPU       XPU     RKNPU       APU       Any       Unk\r\n",
      "                               anchor_generator                                       Y                                                                                \r\n",
      "                                     batch_norm                   Y                   Y                                                                                \r\n",
      "                                       box_clip                                       Y                                                                                \r\n",
      "                                      box_coder                                       Y         Y                                                                      \r\n",
      "                          collect_fpn_proposals                                       Y                                                                                \r\n",
      "                                         concat                   Y         Y         Y         Y                                                                      \r\n",
      "                                         conv2d                   Y         Y         Y         Y         Y                                                            \r\n",
      "                       distribute_fpn_proposals                                       Y                                                                                \r\n",
      "                                elementwise_add                   Y         Y         Y         Y         Y                                                            \r\n",
      "                                elementwise_div                                       Y                                                                                \r\n",
      "                                           feed         Y                   Y                             Y                                                            \r\n",
      "                                          fetch         Y                                                 Y                                                            \r\n",
      "                                         gather                   Y                   Y                                                                                \r\n",
      "                             generate_proposals                                       Y                                                                                \r\n",
      "                                      lod_reset                                       Y                                                                                \r\n",
      "                                            mul                   Y         Y         Y                                                                                \r\n",
      "                                 multiclass_nms         Y                                                 Y                                                            \r\n",
      "                                 nearest_interp                             Y         Y         Y                                                                      \r\n",
      "                                         pool2d                   Y         Y         Y         Y         Y                                                            \r\n",
      "                                           relu                   Y         Y         Y         Y                                                                      \r\n",
      "                                       reshape2         Y         Y                             Y                                                                      \r\n",
      "                                      roi_align                                       Y                                                                                \r\n",
      "                                          scale                   Y         Y         Y         Y         Y                                                            \r\n",
      "                                sequence_expand                                       Y                                                                                \r\n",
      "                                        sigmoid                                       Y         Y                                                                      \r\n",
      "                                          slice                   Y                   Y         Y                                                                      \r\n",
      "                                        softmax                   Y         Y         Y                                                                                \r\n",
      "Paddle-Lite supports this model!\r\n"
     ]
    }
   ],
   "source": [
    "!paddle_lite_opt --print_model_ops=true  --model_dir=inference_model/faster_rcnn_r50_fpn --valid_targets=arm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 转化模型为Paddle-Lite格式\n",
    "如果最后log打印出`Save the optimized model into :XXXsuccessfully`代表模型成功转换，而转换faster_rcnn_r50_fpn模型时会出现转换失败报错：\n",
    "```\n",
    "F0706 21:26:06.841753  1475 kernel.cc:44] Check failed: type no type registered for kernel [generate_proposals/def] output argument [RpnRoisLod]\n",
    "*** Check failure stack trace: ***\n",
    "Aborted (core dumped\n",
    "```\n",
    "因此模型部署时只使用转换后的YoloV3模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 该手机型号是arm_v8架构，只使用CPU的话在valid_targets中只设置arm\r\n",
    "!paddle_lite_opt \\\r\n",
    "    --model_file=inference_model/yolov3_mobilevetv3/__model__ \\\r\n",
    "    --param_file=inference_model/yolov3_mobilevetv3/__params__ \\\r\n",
    "    --optimize_out_type=naive_buffer \\\r\n",
    "    --optimize_out=mobile_cpu_model \\\r\n",
    "    --valid_targets=arm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 使用华为NPU预测时的设置\r\n",
    "!paddle_lite_opt \\\r\n",
    "    --model_file=inference_model/yolov3_mobilevetv3/__model__ \\\r\n",
    "    --param_file=inference_model/yolov3_mobilevetv3/__params__ \\\r\n",
    "    --optimize_out_type=naive_buffer \\\r\n",
    "    --optimize_out=mobile_npu_model \\\r\n",
    "    --valid_targets=npu,arm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!paddle_lite_opt \\\n",
    "    --model_file=inference_model/yolov3_darknet53/__model__ \\\n",
    "    --param_file=inference_model/yolov3_darknet53/__params__ \\\n",
    "    --optimize_out_type=naive_buffer \\\n",
    "    --optimize_out=npu_model \\\n",
    "    --valid_targets=npu,arm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!paddle_lite_opt \\\n",
    "    --model_file=inference_model/yolov3_darknet53/__model__ \\\n",
    "    --param_file=inference_model/yolov3_darknet53/__params__ \\\n",
    "    --optimize_out_type=naive_buffer \\\n",
    "    --optimize_out=cpu_model \\\n",
    "    --valid_targets=arm "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 准备Android Studio开发环境\n",
    "## 安装 Android Studio\n",
    "参考[安装 Android Studio](https://developer.android.google.cn/studio/install),下载自己PC对应的环境安装。\n",
    "本项目介绍在Ubuntu 18.04 64-bit上的Android Studio安装和配置。\n",
    "1. 首先安装64位计算机所需的库\n",
    "```shell\n",
    "sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 lib32z1 libbz2-1.0:i386\n",
    "```\n",
    "> 注意：官方教程没有提到的是，运行Android Studio需要Java环境，而且最新的OpenJDK 11会出现报错，因此需要下载OpenJDK 8\n",
    "确认一下Java版本\n",
    "```shell\n",
    "java --version\n",
    "```\n",
    "下载OpenJDK 8\n",
    "```shell\n",
    "sudo apt-get install openjdk-8-jdk\n",
    "```\n",
    "如果需要，输入对应选项的数字切换Java版本\n",
    "```\n",
    "update-alternatives --config java\n",
    "```\n",
    "```shell\n",
    "There are 2 choices for the alternative java (providing /usr/bin/java).\n",
    "\n",
    "  Selection    Path                                            Priority   Status\n",
    "------------------------------------------------------------\n",
    "  0            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      auto mode\n",
    "  1            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      manual mode\n",
    "* 2            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081      manual mode\n",
    "\n",
    "Press <enter> to keep the current choice[*], or type selection number: 2\n",
    "```\n",
    "\n",
    "此外，部署Paddle模型还需要安装cmake和ninja环境等\n",
    "```shell\n",
    "# 1. Install basic software 参考官方文档，注意权限，应该是root用户\n",
    "apt update\n",
    "apt-get install -y --no-install-recommends \\\n",
    "  gcc g++ git make wget python unzip adb curl\n",
    "\n",
    "# 2. Install cmake 3.10 or above 参考官方文档，注意权限，应该是root用户\n",
    "wget -c https://mms-res.cdn.bcebos.com/cmake-3.10.3-Linux-x86_64.tar.gz && \\\n",
    "    tar xzf cmake-3.10.3-Linux-x86_64.tar.gz && \\\n",
    "    mv cmake-3.10.3-Linux-x86_64 /opt/cmake-3.10 && \\  \n",
    "    ln -s /opt/cmake-3.10/bin/cmake /usr/bin/cmake && \\\n",
    "    ln -s /opt/cmake-3.10/bin/ccmake /usr/bin/ccmake\n",
    "# 3. Install ninja-build\n",
    "sudo apt-get install ninja-build\n",
    "```\n",
    "2. 将下载的 .zip 文件解压缩到相应位置，例如 /usr/local/ 中（对于用户个人资料）或者 /opt/ 中（对于共享用户）。\n",
    "3. 打开一个终端，导航至 android-studio/bin/ 目录，并执行 studio.sh，启动 Android Studio。\n",
    "4. 选择是否想要导入之前的 Android Studio 设置，然后点击 OK。\n",
    "5. Android Studio 设置向导将指导您完成余下的设置步骤，包括下载开发所需的 Android SDK 组件。\n",
    "\n",
    "> 参考安装视频：\n",
    "[Ubuntu上推荐的设置流程](https://developer.android.google.cn/studio/videos/studio-install-linux.mp4?hl=zh-cn)\n",
    "\n",
    "> 注意：安装过程中会如果出现下面的报错`unable to access android sdk add-on lis`，点击`cancel`跳过。\n",
    "\n",
    "![file](https://ss0.bdstatic.com/70cFuHSh_Q1YnxGkpoWK1HF6hhy/it/u=1251595712,1750168340&fm=26&gp=0.jpg)\n",
    "\n",
    "## 导入Paddle-Lite-Demo\n",
    "在自己的PC上下载好Paddle-Lite-Demo项目\n",
    "```shell\n",
    "git clone https://gitee.com/paddlepaddle/Paddle-Lite-Demo.git\n",
    "```\n",
    "\n",
    "在`PaddleLite-android-demo`目录下找到准备导入的`yolo_detection_demo`，如下图所示\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/afbaa1397f28434fa6f433f306deedb84289d1c3936c4c50b1289b1dd96c743b)\n",
    "\n",
    "在Android Studio的欢迎界面点击`Open an existing Android Studio project`导入`yolo_detection_demo`工程\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/b87ae9237c7c4af4926dc2f57d8442137a73aeaaac6a4074a6257d034dca04a4)\n",
    "## 配置SDK和NDK\n",
    "进入开发界面后，点击右上角的`SDK Manager`按钮，下载好SDK和NDK\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/66e5e66c920844eaa73300ede4ea956848c84b3956d143c28a6908b45cac522e)\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/cc377161863647fc8cd22f51ad4d740731764c90b01349a5bb400897f20777e6)\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/306346149f75472a88824fb157b5b690bb7d256aca9e449a981626d6b6f5a14f)\n",
    "\n",
    "使用快捷键`Ctrl+Alt+Shift+S`打开`Project Structure`设置好该项目的SDK和NDK路径\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/6accffab2545421caa74fa8e854a5bde95d0076332054160bbfe0f9e7be2c918)\n",
    "\n",
    "## 修改`build.gradle`，配置国内镜像仓库\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/5a2cb44c2237443da816de6b6abe58afb3ca34bae6114ba89977f75ef04d4481)\n",
    "\n",
    "这里其实就是将原工程`build.gradle`文件中的\n",
    "```java\n",
    "    repositories {\n",
    "        google()\n",
    "        jcenter()\n",
    "        \n",
    "    }\n",
    "```\n",
    "全部替换成对应的国内镜像加速仓库，修改后文件如下\n",
    "```java\n",
    "// Top-level build file where you can add configuration options common to all sub-projects/modules.\n",
    "\n",
    "buildscript {\n",
    "    repositories {\n",
    "        maven { url 'https://maven.aliyun.com/repository/google/' }\n",
    "        maven { url 'https://maven.aliyun.com/repository/jcenter/'}\n",
    "        \n",
    "    }\n",
    "    dependencies {\n",
    "        classpath 'com.android.tools.build:gradle:3.4.0'\n",
    "        \n",
    "        // NOTE: Do not place your application dependencies here; they belong\n",
    "        // in the individual module build.gradle files\n",
    "    }\n",
    "}\n",
    "\n",
    "allprojects {\n",
    "    repositories {\n",
    "        maven { url 'https://maven.aliyun.com/repository/google/' }\n",
    "        maven { url 'https://maven.aliyun.com/repository/jcenter/'}\n",
    "        \n",
    "    }\n",
    "}\n",
    "\n",
    "task clean(type: Delete) {\n",
    "    delete rootProject.buildDir\n",
    "}\n",
    "```\n",
    "## 重新`build`项目\n",
    "个人的做法是关闭当前工程再重新进入（感觉比较快），或者也可以使用快捷键`Ctrl+F9`，等待进度条跑完，最终要如同官方文档的介绍，`Build`中`Sync`要全部同步完成，看到绿色打勾才行。\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/9cc3055c40ae433486ae3aa4a870e2f300bb1bab834144f4b61a2d33c3c031c2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 部署模型到移动端\n",
    "## 手机开发者模式连接\n",
    "1. 将手机通过USB数据线连接到PC上\n",
    "2. 在手机上点击：“设置”——“关于手机”——“版本号”（连续多次点击版本号才能启动开发者模式）\n",
    "\n",
    " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/ae38fa53cfb44ac7a2cceee219cca9e3f78e8a47dd4b46bcbaecca48249e4a66\" width = \"300\" height = \"200\" alt=\"yolov3_darknet53_garbage_for_npu\" align=center />\n",
    "\n",
    "3. 回到“设置”界面，可以在“搜索设置项”中查找“开发人员选项”并进入\n",
    "4. **重点：先打开“仅充电模式下允许ADB调试”然后再打开“USB调试”——否则Android Studio识别不到华为手机**\n",
    "\n",
    " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/551711d636ec486483c45ddea968d16f6a4c2ccb538f4b73a502b0a687a81704\" width = \"300\" height = \"200\" alt=\"yolov3_darknet53_garbage_for_npu\" align=center />\n",
    "\n",
    "## 部署demo的CPU和NPU模型\n",
    "1. 完成构建后，`PaddleLite-android-demo/yolo_detection_demo/app/src/main/assets`目录下会出现COCO数据集上的`yolov3_mobilenet_v3_for_cpu`和`yolov3_mobilenet_v3_for_hybrid_cpu_npu`两个demo模型。\n",
    "\n",
    "2. 如果直接点击`run`，会将目标检测的CPU模型`yolov3_mobilenet_v3_for_cpu`直接部署到手机。\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/b7e28a5cd430499eae3503994911bdcc1ecb11d8c8eb4d96a518c7b5dd0cbf21)\n",
    "\n",
    "3. 如果需要部署NPU模型，那么需要修改`PaddleLite-android-demo/yolo_detection_demo/app/src/main/res/values/strings.xml`中对应的`MODEL_DIR_DEFAULT`，将其改为`yolov3_mobilenet_v3_for_hybrid_cpu_npu`所在位置，如下图\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/8d99117dd4c144528a25796e3200a3e05a5ac2e511f349a2abe00db1a0e56b92)\n",
    "\n",
    "4. demo中NPU模型预测效果：\n",
    "\n",
    " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/7b48df24bd8a4b0cb41ff1f6dc6321f29043a98e5e5c4329ae1f12b8bb0c011d\" width = \"300\" height = \"200\" alt=\"yolov3_darknet53_garbage_for_npu\" align=center />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 部署迁移学习模型\n",
    "部署迁移学习模型其实只比前面部署**NPU上的demo模型多一个步骤**，就是修改对应label文件的路径：\n",
    "1. 以`yolov3_darknet53_for_cpu`模型为例，可以在`PaddleLite-android-demo/yolo_detection_demo/app/src/main/assets/models`目录下新建一个放该模型的文件夹`yolo_v3_garbage_for_cpu`，然后将下载好的`cpu_model.nb`模型放进去并重命名为`model.nb`\n",
    "2. 在`PaddleLite-android-demo/yolo_detection_demo/app/src/main/assets/labels`目录下仿照`coco-labels-2014_2017.txt`新建一个`garbage-labels.txt`内容如下\n",
    "```\n",
    "background\n",
    "green\n",
    "blue\n",
    "transparent\n",
    "white\n",
    "yellow\n",
    "red\n",
    "```\n",
    "3. 修改`PaddleLite-android-demo/yolo_detection_demo/app/src/main/res/values/strings.xml`中对应的`MODEL_DIR_DEFAULT`，将其改为`models/yolov3_darknet53_garbage_for_cpu`\n",
    "4. 修改`PaddleLite-android-demo/yolo_detection_demo/app/src/main/res/values/strings.xml`中对应的`LABEL_PATH_DEFAULT`，将其改为`labels/garbage-labels.txt`\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/873c32bf3b3d47a6af896c4f148cdbe9e22ab42d9bd74510b25ba3661b6127d1)\n",
    "## 预测效果\n",
    "- `yolov3_mobilenet_v3_garbage_for_npu`:\n",
    "\n",
    " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/30064752075a42f68e2c32dd962a3a51e23f697cd96647b4a03c226cf40c24e6\" width = \"300\" height = \"200\" alt=\"yolov3_darknet53_garbage_for_npu\" align=center />\n",
    "\n",
    "- `yolov3_darknet53_garbage_for_npu`:\n",
    "\n",
    " <img src=\"https://ai-studio-static-online.cdn.bcebos.com/4245953d9aed479a8cc2ea0466eed67fd84c2390675745b5a574fc89ba83bf69\" width = \"300\" height = \"200\" alt=\"yolov3_darknet53_garbage_for_npu\" align=center />\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#  小结\n",
    "总结了一些使用PaddleX+Paddle-Lite部署遇到的问题，待进一步研究或者提issue\n",
    "- PaddleX的deploy模块直接转化paddle-lite模型会报错\n",
    "- Paddle-Lite源码编译失败\n",
    "- 非armv8架构需要编译Android预测库\n",
    "- 目前目标检测模型只成功部署了YoloV3，FasterRCNN会报错，DCN算子Lite尚不支持"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PaddlePaddle 1.8.0 (Python 3.5)",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
